More recently, Newton-type methods using sparsified Hessian matrices have demonstrated promising results on OT computation, but there still remain a lot of unresolved open questions.

These methods can be broadly categorized into two types: function learning and operator learning approaches. In function learning, the goal is to directly learn the solution.

However, these methods typically reach superlinear convergence only when the stochastic Hessian noise diminishes, increasing per-iteration costs over time.

Empirically, we verify the computational efficiency of our methods.

Multitask reinforcement learning (MTRL) suffers from scalability issues when the number of tasks or trajectories grows large.

Second, in the non-parametric machine learning setting, we provide an explicit algorithm combining the previous scheme with Nyström projection techniques, andprovethatitachievesoptimal generalization bounds with atime complexity of orderO(ndfλ), a memory complexity of orderO(df2λ) and no dependence on the condition number, generalizing the results known for leastsquaresregression.Here nisthenumberofobservationsand dfλ istheassociated degrees of freedom.

Latter approach was a breakthrough in 1990s, it lead to interior-point methods.

We develop a randomized Newton method capable of solving learning problems with huge dimensional feature spaces, which is a common setting in applications such as medical imaging, genomics and seismology. Our method leverages randomized sketching in a new way, by finding the Newton direction constrained to the space spanned by a random sketch. We develop a simple global linear convergence theory that holds for practically all sketching techniques, which gives the practitioners the freedom to design custom sketching approaches suitable for particular applications. We perform numerical experiments which demonstrate the efficiency of our method as compared to accelerated gradient descent and the full Newton method. Our method can be seen as a refinement and a randomized extension of the results of Karimireddy, Stich, and Jaggi (2019).